Classification Under Misspecification: Halfspaces, Generalized Linear Models, and Connections to Evolvability

In this paper we revisit some classic problems on classification under misspecification. In particular, we study the problem of learning halfspaces under Massart noise with rate $\eta$. In a recent work, Diakonikolas, Goulekakis, and Tzamos resolved a long-standing problem by giving the first efficient algorithm for learning to accuracy $\eta + \epsilon$ for any $\epsilon > 0$. However, their algorithm outputs a complicated hypothesis, which partitions space into $\text{poly}(d,1/\epsilon)$ regions. Here we give a much simpler algorithm and in the process resolve a number of outstanding open questions: (1) We give the first proper learner for Massart halfspaces that achieves $\eta + \epsilon$. We also give improved bounds on the sample complexity achievable by polynomial time algorithms. (2) Based on (1), we develop a blackbox knowledge distillation procedure to convert an arbitrarily complex classifier to an equally good proper classifier. (3) By leveraging a simple but overlooked connection to evolvability, we show any SQ algorithm requires super-polynomially many queries to achieve $\mathsf{OPT} + \epsilon$. Moreover we study generalized linear models where $\mathbb{E}[Y|\mathbf{X}] = \sigma(\langle \mathbf{w}^*, \mathbf{X}\rangle)$ for any odd, monotone, and Lipschitz function $\sigma$. This family includes the previously mentioned halfspace models as a special case, but is much richer and includes other fundamental models like logistic regression. We introduce a challenging new corruption model that generalizes Massart noise, and give a general algorithm for learning in this setting. Our algorithms are based on a small set of core recipes for learning to classify in the presence of misspecification. Finally we study our algorithm for learning halfspaces under Massart noise empirically and find that it exhibits some appealing fairness properties.Comment: 51 pages, comments welcom

A New Approach to Learning Linear Dynamical Systems

Linear dynamical systems are the foundational statistical model upon which control theory is built. Both the celebrated Kalman filter and the linear quadratic regulator require knowledge of the system dynamics to provide analytic guarantees. Naturally, learning the dynamics of a linear dynamical system from linear measurements has been intensively studied since Rudolph Kalman's pioneering work in the 1960's. Towards these ends, we provide the first polynomial time algorithm for learning a linear dynamical system from a polynomial length trajectory up to polynomial error in the system parameters under essentially minimal assumptions: observability, controllability, and marginal stability. Our algorithm is built on a method of moments estimator to directly estimate Markov parameters from which the dynamics can be extracted. Furthermore, we provide statistical lower bounds when our observability and controllability assumptions are violated

Online and Distribution-Free Robustness: Regression and Contextual Bandits with Huber Contamination

In this work we revisit two classic high-dimensional online learning problems, namely linear regression and contextual bandits, from the perspective of adversarial robustness. Existing works in algorithmic robust statistics make strong distributional assumptions that ensure that the input data is evenly spread out or comes from a nice generative model. Is it possible to achieve strong robustness guarantees even without distributional assumptions altogether, where the sequence of tasks we are asked to solve is adaptively and adversarially chosen? We answer this question in the affirmative for both linear regression and contextual bandits. In fact our algorithms succeed where conventional methods fail. In particular we show strong lower bounds against Huber regression and more generally any convex M-estimator. Our approach is based on a novel alternating minimization scheme that interleaves ordinary least-squares with a simple convex program that finds the optimal reweighting of the distribution under a spectral constraint. Our results obtain essentially optimal dependence on the contamination level $\eta$, reach the optimal breakdown point, and naturally apply to infinite dimensional settings where the feature vectors are represented implicitly via a kernel map.Comment: 66 pages, 1 figure, v3: refined exposition and improved rate

Enhanced hippocampal long-term potentiation and spatial learning in aged 11ß-hydroxysteroid dehydrogenase type 1 knock-out mice

Glucocorticoids are pivotal in the maintenance of memory and cognitive functions as well as other essential physiological processes including energy metabolism, stress responses, and cell proliferation. Normal aging in both rodents and humans is often characterized by elevated glucocorticoid levels that correlate with hippocampus-dependent memory impairments. 11ß-Hydroxysteroid dehydrogenase type 1 (11ß-HSD1) amplifies local intracellular ("intracrine") glucocorticoid action; in the brain it is highly expressed in the hippocampus. We investigated whether the impact of 11ß-HSD1 deficiency in knock-out mice (congenic on C57BL/6J strain) on cognitive function with aging reflects direct CNS or indirect effects of altered peripheral insulin-glucose metabolism. Spatial learning and memory was enhanced in 12 month "middle-aged" and 24 month "aged" 11ß-HSD1<sup>–/–</sup> mice compared with age-matched congenic controls. These effects were not caused by alterations in other cognitive (working memory in a spontaneous alternation task) or affective domains (anxiety-related behaviors), to changes in plasma corticosterone or glucose levels, or to altered age-related pathologies in 11ß-HSD1<sup>–/–</sup> mice. Young 11ß-HSD1<sup>–/–</sup> mice showed significantly increased newborn cell proliferation in the dentate gyrus, but this was not maintained into aging. Long-term potentiation was significantly enhanced in subfield CA1 of hippocampal slices from aged 11ß-HSD1<sup>–/–</sup> mice. These data suggest that 11ß-HSD1 deficiency enhances synaptic potentiation in the aged hippocampus and this may underlie the better maintenance of learning and memory with aging, which occurs in the absence of increased neurogenesis

A model-based cost-utility analysis of multi-professional simulation training in obstetric emergencies

ObjectiveTo determine the cost-utility of a multi-professional simulation training programme for obstetric emergencies-Practical Obstetric Multi-Professional Training (PROMPT)-with a particular focus on its impact on permanent obstetric brachial plexus injuries (OBPIs).DesignA model-based cost-utility analysis.SettingMaternity units in England.PopulationSimulated cohorts of individuals affected by permanent OBPIs.MethodsA decision tree model was developed to estimate the cost-utility of adopting annual, PROMPT training (scenario 1a) or standalone shoulder dystocia training (scenario 1b) in all maternity units in England compared to current practice, where only a proportion of English units use the training programme (scenario 2). The time horizon was 30 years and the analysis was conducted from an English National Health Service (NHS) and Personal Social Services perspective. A probabilistic sensitivity analysis was performed to account for uncertainties in the model parameters.Main outcome measuresOutcomes for the entire simulated period included the following: total costs for PROMPT or shoulder dystocia training (including costs of OBPIs), number of OBPIs averted, number of affected adult/parental/dyadic quality adjusted life years (QALYs) gained and the incremental cost per QALY gained.ResultsNationwide PROMPT or shoulder dystocia training conferred significant savings (in excess of £1 billion ($1.5 billion)) compared to current practice, resulting in cost-savings of at least £1 million ($1.5 million) per any type of QALY gained. The probabilistic sensitivity analysis demonstrated similar findings.ConclusionIn this model, national implementation of multi-professional simulation training for obstetric emergencies (or standalone shoulder dystocia training) in England appeared to both be cost-saving when evaluating their impact on permanent OBPIs